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Abstract. In a recent work, the combinatorial interpretation of the polynomial a(n; ki, k%, ■ • ■ , k n ) 
counting the number of Monotone Triangles with bottom row k\ < &2 < ■ • • < k n was extended 
to weakly decreasing sequences k\ > k2 > ■ ■ ■ > k n . In this case the evaluation of the polynomial 
is equal to a signed enumeration of objects called Decreasing Monotone Triangles. In this paper 
we define Generalized Monotone Triangles - a joint generalization of both ordinary Monotone 
Triangles and Decreasing Monotone Triangles. As main result of the paper we prove that the 
evaluation of a(n\ k\, k2, . . . , k n ) at arbitrary (fci, £2, . . . , k n ) £ Z n is a signed enumeration of 
Generalized Monotone Triangles with bottom row [ki, k2, . . . , k n ). Computational experiments 
indicate that certain evaluations of the polynomial at integral sequences yield well-known round 
numbers related to Alternating Sign Matrices. The main result provides a combinatorial inter- 
pretation of the conjectured identities and could turn out useful in giving a bijective proof. 



l. Introduction 

A Monotone Triangle of size n is a triangular array of integers (a-i,j)i<j<i<-r 

ai,i 

0-2,1 0,2,2 



On,l ' ' ' O n n 

with strict increase along rows and weak increase along North-East- and South-East-diagonals, i.e. 
ijj < cJij+i, <H+i,j < a i,j ^ Oi+ij+i- An example of a Monotone Triangle of size 5 is given in 



FiglU 



4 

4 5 
3 5 7 
2 5 6 8 
2 4 5 8 9 



Figure 1. One of the 16939 Monotone Triangles with bottom row (2, 4, 5, 8, 9). 



Key words and phrases. Combinatorial Reciprocity. Monotone Triangle, Generalized Monotone Triangle, Alter- 
nating Sign Matrix. 

Supported by the Austrian Science Foundation FWF, START grant Y463. 

Fakultat fur Mathematik, Universitat Wien, Nordbergstrafie 15, A-1090 Wien, Austria. 

E-M ail : lukas .riegler@univie.ac. at. 

1 



2 



LUKAS RIEGLER 



For each n > 1, there exists a unique polynomial a(n; k%, fe, . . . , k n ) of degree n — 1 in each 
of the n variables such that the evaluation of this polynomial at strictly increasing sequences 
k\ < hi < ■ ■ ■ < k n is equal to the number of Monotone Triangles with prescribed bottom row 
(fci, k%, . . . , k n ) - for example a(5; 2, 4, 5, 8, 9) = 16939. This result was derived in [FisOl], where 
the polynomials are given explicitly in terms of an operator formula. 

In [FRllj we studied the evaluation of a(n; k\, . . . , k n ) at weakly decreasing sequences k\ > fc 2 > 
• • • > k n . It turned out that the evaluation can be interpreted as signed enumeration of the following 
combinatorial objects: 

A Decreasing Monotone Triangle (DMT) of size n is a triangular array of integers (<Xi,j)i<j<i< n 
having the following properties: 

• The entries along North-East- and South-East-diagonals are weakly decreasing. 

• Each integer appears at most twice in a row. 

• Two consecutive rows do not contain the same integer exactly once. 

One of the motivations for considering evaluations of a(n; fci, . . . , k n ) at non- increasing (fci, . . . , k„) £ 
Z ra stems from the connection to Alternating Sign Matrices. An Alternating Sign Matrix (ASM) of 
size n is a n X n-matrix with entries in {0, 1, —1} such that in each row and column the non-zero 
entries alternate in sign and sum up to 1. It is well-known that the set of ASMs is in bijection with 
the set of Monotone Triangles with bottom row (1,2, ... ,n). Counting the number of ASMs of size 
n had been an open problem for more than a decade until the first two independent proofs were 
given by D. Zeilberger f [Zei96j ) and G. Kuperberg (j Kup96| ) in 1996 (see B:e99| for more details). 
The Refined ASM Theorem - i.e. the refined enumeration with respect to the unique 1 in the first 
row - was reproven by I. Fischer in 2007 ( Fis07 ). The identity 

a(n;ki,...,k n ) = (-l)" _1 a(n; k 2 , . . . , k n , k\ —n) (1.1) 

plays one of the key roles in this algebraic proof. A bijective proof of (jl.lj) could give more 
combinatorial insight to the theorem. However, note that ii k\ < k 2 < ■ ■ ■ < k n , then k n > k\ — n, 
i.e. (jl.ll) can per se only be understood as identity satisfied by the polynomial. 

The objective of this paper is to give an interpretation to the evaluation of a(n;k\, . . . ,k n ) at 
arbitrary (k\, . . . , k n ) € Z". For this, we define triangular arrays of integers which locally combine 
the restrictions of ordinary Monotone Triangles and Decreasing Monotone Triangles: 

A Generalized Monotone Triangle (GMT) is a triangular array (a» ,j)i<j<i<n of integers satisfying 
the following conditions: 

(1) Each entry is weakly bounded by its SW- and SE-neighbour, i.e. 

min{ai+i,j,a< + i ) j + i} < a hj < max{a i+ i j; ,-, a, + i )J+ i}. 

(2) If three consecutive entries in a row are weakly increasing, then their two interlaced neigh- 
bours in the row above are strictly increasing, i.e. 

a i+l,j < <H+l,j+l < a i+l,j+2 O-i.j < a Lj + l- 

(3) If two consecutive entries in a row are strictly decreasing and their interlaced neighbour 
in the row above is equal to its SW-/SE- neighbour, then the interlaced neighbour has a 



GENERALIZED MONOTONE TRIANGLES 



3 



left/right neighbour and is equal to it, i.e. 

a iJ = a i+l,j > Oi+lJ+1 — ^ a i,j-l = a i,ji 
a i+l,j > Q>i+l,j+l — a i,j a i,j + l — a i,j- 



By way of illustration, let us find all GMTs with bottom row (4, 2, f , 3): First, construct all possible 
penultimate rows (h, h, h)- Condition ([T]) implies that h € {2, 3, 4}, Condition <j3j> further restricts 
it to l\ G {2, 3}. If on the one hand l\ — 2, then Condition ([3]) forces 1% = 2. The right-most entry 
1-3 is bounded by 1 and 3, but actually l± = h = h = 2 would violate Condition @, so ^3 6 {1,3}. 
If on the other hand = 3, then Condition ([3]) implies that I2 = h = 1- Continuing in the same 
way with all penultimate rows yields the four GMTs depicted in Figure [2j 

2 2 
2 2 2 3 

2 2 1 2 2 3 

4 2 1 3 4 2 1 3 

3 1 

2 3 11 
2 2 3 3 1 1 

4 2 1 3 4 2 1 3 



FIGURE 2. The four GMTs with bottom row (4, 2, 1, 3). 



For fei < &2 < • • • < k n , the set of GMTs with bottom row (k±, . . . , k n ) is equal to the set of 
Monotone Triangles with this bottom row: Every GMT with strictly increasing bottom row is by 
conditions ([lj and (|2|) a Monotone Triangle. Conversely, the weak increase along NE- and SE- 
diagonals of Monotone Triangles implies condition (TTJ) of GMTs, the strict increase condition ©, 
and the premise of ([3]) can not hold. 

For k\ > &2 > ■ • • > k n , the set of GMTs with bottom row (k±, . . . , k n ) is equal to the set of 
Decreasing Monotone Triangles with this bottom row: The NE- and SE-diagonals of every GMT 
with weakly decreasing bottom row are by condition JT]) weakly decreasing. This also implies a weak 
decrease along rows, and thus three consecutive equal entries in a row would contradict condition ([2]) . 
Furthermore, two consecutive rows containing an integer exactly once would contradict condition 
([3]). Conversely, the weak decrease of DMTs along NE- and SE-diagonals implies condition ([T]) 
and weak decrease along rows. Thus, the premise of (|2|) can only hold if three consecutive entries 
coincide, which is not admissible in DMTs. Finally, condition ([3|) follows from the weak decrease 
along rows together with the condition that two consecutive rows do not contain the same entry 
exactly once. 

Therefore, Generalized Monotone Triangles are indeed a joint generalization of ordinary Monotone 
Triangles and Decreasing Monotone Triangles. The main result of the paper is that the evaluation 
a(n; k\, . . . , k n ) is a signed enumeration of the GMTs with bottom row (fci, fe, ■ ■ ■ , k n ). The sign 
of a GMT is determined by the following two statistics: 

(1) An entry is called newcomer if a i+1 j > a^j > aj+ij+i. 
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(2) A pair (x, x) of two consecutive equal entries in a row is called sign- changing ; if their 
interlaced neighbour in the row below is also equal to x. 

Let G n (ki, k2, . . . , k n ) denote the set of GMTs with bottom row (k±, ki, ■ ■ ■ , k n ). 

Theorem 1. Let n > 1 and (ki, &2, ■ ■ • , k n ) € Z n . Then 

a(n;h,...,k n )= £ (-l) sc ^, 

where sc(A) is the total number of newcomers and sign-changing pairs in A. 
Applying Theorem Q] to our example in Figure [2] yields a (4 ; 4, 2, 1,3) = —2. 

Theorem [1] is known to be true for strictly increasing sequences k\ < fc 2 < ■ ■ ■ < k n , as in this case 
the set G n {ki, ■ ■ ■ , k n ) is equal to the set of Monotone Triangles with bottom row (fci, fc 2 , ■ ■ ■ , k n ) 
and sc(A) = for every Monotone Triangle. 

Lemma 3 of |FR11] implies the correctness of Theorem [T] for weakly decreasing bottom rows: In this 
case G n (ki, . . . , k n ) is equal to the set of DMTs with bottom row (fci, . . . , k n ) and the sc-functions 
coincide. K. Jochemko and R. Sanyal recently gave a proof of the theorem in this case from a 
geometric point of view ( |JS12j ). 

In Section[5]we give a straight-forward proof of Theorem[T]using a recursion satisfied by a(n; ki, . . . , k n ). 
In Section [3] a connection with a known generalization f |Fisllj ) is established, which enables us to 
give a shorter, more subtle proof of Theorem [T] Apart from being a joint generalization of Mono- 
tone Triangles and DMTs, this generalization is more reduced in the sense that fewer cancellations 
occur in the signed enumerations than in previously known generalizations. In Section 4 we apply 
the theorem to give a combinatorial proof of an identity satisfied by ct(n; k\, . . . , k n ) and provide a 
collection of open problems. 



2. Proof of Theorem \T\ 

The number of Monotone Triangles with bottom row (ki, . . . , k n ) can be counted recursively by de- 
termining all admissible penultimate rows . . . , l n -i) and summing over the number of Monotone 
Triangles with these bottom rows. The polynomial a(n; k\, . . . , k n ) hence satisfies 

a(n;ki, . . . ,k n ) = ^ a(n - 1; 1%, . . . , l n -i) ( 2 -l) 

(li,...,J„-i)ez n- \ 
ki<h<k 2 <l2<—<k„-i<l n -i<k„, 
h<h+i 

(fci,...,fc„) 

for all ki < &2 < • " • < km ki 6 Z. In fact ([Fis06]), one can define a summation operator 

(h,...,ln-l) 

for arbitrary (k±, . . . , k n ) G Z" such that 

(fei,...,fe„) 

a(n; k x , . . . , k n ) = ^ a ( n ~ ^ h, ■ ■ ■ > l n -l) ( 2 -2) 
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holds. The summation operator is defined recursively for arbitrary (fci, . . . , k n ) 6 Z n : 
(fci,...,fe n ) (fci,...,fc„_i) k n 

A (h, ■ ■ -Jn-l) ■= E Mh,...,ln-2,ln-l) (2.3) 

(il,...,i„_l) (Jl..".2n-2) Jn-I=fcn-1 + 1 

(ki....,k n -2,k n — 1 — 1) 

+ A(Zi,...,/ n _2,fc„-i), n > 2, 

(Hi,...,i„_ a ) 

(fcl) 

with Y^, := id and the extended definition of simple sums 

C) 

b [ 0, 6 = a — 1, 

-fi'/m. » + i<-i. (2 - 4) 



Using induction and (|2.1j) . it is clear that (|2.2p holds for increasing sequences k\ < &2 < ■ ■ ■ < k n . 

(ki,...,k n ) 

To prove it for arbitrary Qci, . . . , k n ) G Z ra , let us first note that applying Y to a polynomial 

(h,...,ln-l) 

in (Zi, . . . , Z„_i) yields a polynomial in (A»i, . . . , k n ): In the base case n = 2, write the polynomial 
p(Zi) in terms of the binomial basis p(Zi) = E™=o Ci Ci ) * polynomial q(x) := Y17=i Ci ~ 1 (T) then 
satisfies q(x + l) — q(x) — p(x). For integers a < b, it follows that X^ 1= a-P(M = ?(^ + 1) — ?( a )> but 



this is by definition (|2.4I) true for arbitrary o,f) £ Z. The inductive step is immediate using (|2.3p . 
Thus, we know that the right-hand side of (|2.2[) is a polynomial in (fci, . . . , fc„) coinciding with the 
polynomial on the left-hand side whenever k\ < k% < • • • < k n . Since a polynomial in n variables 
is uniquely determined by these values, it follows that (|2.2p indeed holds. The same is true for the 
alternative recursive description 

(ki,...,fc„) (fci,...,fc„_i) fc n 

A(k,...,l n -i) = E £ A(Zi,...,Z n _ 2 ,Zn-i) (2.5) 

(*!,.. .,i„-l) (Jl,-.in-a) i„-l=fc„-l 

(fe 1 ,...,fc„_ 2 ) 

- A(Zi,...,Z„_3,fc„_i,fc„_i) ) n > 3. 

(il,...,i„- 3 ) 

The following Lemma establishes a connection between the summation operator and GMTs, which 
then gives us the means to prove Theorem [T] inductively. 

Lemma 1. Let V{k\, . . . , k n ) denote the set of (n — l)-st rows of elements in Q„(ki, k%, . . . , k n ). 
Then every function A(Ji, . . . , l n —i) satisfies 

(fcl,...,*n) 

£ A(l 1 ,...,l n _ 1 )= Y, (-l) s « k > l) A(h,..., /„_!), n>2, 

(il,-,2n-l)£'P(fel,...,fen) 

where sc(k; 1) := sc(fei, . . . , k n ; l\, . . . , Z n -i) is Z/ie ioiaZ number of newcomers and sign-changing 
pairs in (1%, . . . , Z n _i). 
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Before proving the Lemma, let us first give a remark, which is solely based on the definition of 
GMTs. In general, the set of admissible values for an entry depends on its neighbours and 
as well as the four adjacent entries fcj_i, fa, fcj+i and fa+ 2 in the row below - ordered 

ki—i ki ki-\.\ k{+2 

- in the following way: If fci_i > i = ki, then the only admissible value is li — ki. Symmetrically, 
if fci+i = > fci+2, then = fci+i. Otherwise, li can take any value strictly between ki and 
fa+i. To determine, whether = fcj is allowed, check whether fej > fcj+i, or fa—% > fc, < fcj+i 
or /cj_i < ki < ki-\-i- If fcj > fci+i, then = fcj is admissible, if and only if = fcj. If 
> ki < ki + i, then ^ = fcj is admissible. If fcj_i < fcj < fci+i, then ^ = fcj is admissible, if and 
only if < fcj. Determining whether ij = fe^+i is admissible works symmetrically. 



Proof. If n = 2, then the result is immediate using (|2.4j) . For n = 3, let us check the case fa >k 2 , 
fa < ^3 (the other cases can be shown in the same way) : 



{k 1 ,k 2 ,k 3 ) (kiM) k 3 (fei) 

E A{hM)= E £ A(h,l 2 )-J2Mk2,k 2 ) 

(h,h) (h) h = k 2 () 

(fei) fe 2 fe 3 fes 

= E E E^i.w+E E ^(fci,M-^2,fc 2 ) 

() i 1 =fc 1 +iz 2 =fc 2 () ; 2 =fc 2 

k 2 k 3 

= E E ^i,^)-A(fc 2 ,fc 2 ). 

l\—k\ l 2 —k 2 



If fc x = fc 2 , then V(fa,fa, fa) = {(fa,l 2 ) : fa < l 2 < fa} with no newcomers or sign-changing pairs. 
If ki > fa, then V(fa,k 2 ,fa) = {(h,h) ■ fa > h > fa, fa < l 2 < k 3 } U {(fc 2 ,fc 2 )}. The entry h is 
either a newcomer or contained in a sign-changing pair. The claimed equation follows from (|2.4p . 

For n > 4, we have to distinguish between the cases fe n _i < k n (Case 1) and fc n _i > k n (Case 
2). The remark preceding the proof also suggests a different behaviour depending on whether l n -\ 
- the rightmost entry of the penultimate row - is equal to fc n _i or not. Indeed, this yields the 
sub-cases 1.1, 1.2 and 2.1, 2.2 respectively. 

Case 1 (k n _i < k n ) : 
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Recursion (|2.3p of the summation operator and the induction hypothesis yield 
Oi,...,fc n ) 

A(h,...,l n -i) 

(h,...,i n -i) 

(*!,.. .,fc n _l) fc„ (*!,.. .,fc n _ 2 ,*„_l-l) 

£ 51 A(Z 1 ,...,i„_ 1 )+ £ A(/i,...,; n _ 2 ,fcn-l) 

(2l,...,J„- a ) 2»-l=fc»-l+l (il,-,«n-a) 

kn 

£ (_i)-c(*i.....*»-iiii.»..i»- a ) £ . . . , i n _i) 

(Zl,...,Z„- 2 )eP(fci,...,fc„_i) i„_i=fe„_i+l 

+ £ (-l) sc ( fcl - fc "- 2 ' fc "- 1 - 1 ^- i "- 2 )A(/ 1 ,...,?„_ 2 ,fc„_ 1 ). 

(ii,...,i„_ 2 )e7'(fci,...,fe„-2,fe„_i-i) 
To see that this is further equal to 

£ (-lr^'Aft,...,^!), 

(Ji,...,«„_i)e7'(fci,...,fc n ) 
let us show that for fe n _i < fc„ 

■P(fci, . . . , fc„) 

= V{k x , kn-l) x {/„_i | fc„_i < /„_i < k„} U . . . , fc„- 2 , K-x - 1) x {fc„_i} (2.6) 

holds, and that each fixed row causes the same total number of sign-changes on the left-hand side 
as on the right-hand side. 



Since the first n — 2 entries of the bottom row are identical on both sides of (|2.6[) . it suffices - by the 
remark preceding the proof - to show that the restrictions imposed on £ n _i, l n -2 and l n — 3 are the 
same on both sides. For this, consider the entry l n -\ and distinguish between < l n -\ < k n 

and Z n _! = fe n _ x : 

Case 1.1 (k n _i < I n _i < k n ) : 

If fc„-2 > fcn-i) then fc„_2 > ln-2 > fcn-i on both sides: 



Left-hand side of (12.6 



I n - 



Is 



> 



k n —i 



Right-hand side of 



Is 



ln-2 



> 



ln-1 



4 



The restrictions for l n s ar e the same on both sides. The entry l n —i does not contribute a sign- 
change, and the entry l n -2 is involved in a sign-change on both sides. 



If fcn-2 < fcn-ij then fc„_ 2 < i n _2 < &n-i 011 both sides: 
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Left-hand side of (EH)!) 



ln-2 In— I 

k-n — 2 k n —l < k n 



Right-hand side of (pO))) 



h 

k n —2 k n —l 



The restrictions for Z„_3 are the same on both sides. The entry does not contribute a sign- 
change, and the entry l n -2 is involved in a sign-change on the left-hand side, if and only if it is on 
the right-hand side. 

It follows that sc(fci, . . . , k n _ 1 ;l 1 , . . . ,i n _ 2 ) = sc(fci, . . . , k n \ h, . . . ,Z„_i). 
Case 1.2 (l„_i = k n _x) : 

If k n -2 — k n -i, then there is no row on the left-hand side with l n -% — k n -i, and on the right-hand 
side this would imply l n s = l n -2 = ln-i = k n -i- 





Left-hand side of 








i 


k n —i 






kn — 2 = kn—1 








Right-hand side of 








k n —i k n —i 




k n — l 






-A 




k n -3 


kn-1 > 


fen-l - 1 





But since a GMT can not contain three consecutive equal entries, such rows are not contained on 
the right-hand side. 

If k n -2 < k n -i — 1, then k n -2 < ln-2 < k n -i on both sides: 



Left-hand side of ( 


12.6) 


ln-2 


k n — l 


U // 




/ v- 




k n —2 k n —l 




Right-hand side of 


(12.61) 


ln-2 


&n — 1 


h 








k n —2 ' k n —i 


- 1 
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The restrictions for l n -z are the same on both sides. The entry l n -\ does not contribute a sign- 
change, and the entry l n -i is involved in a sign-change on the left-hand side, if and only if it is on 
the right-hand side. 

If k n _2 > fc n -i, then k n —% > l n -2 > k n -i on both sides: 



Left-hand side of (12.6 



In— 2 k n — 1 

k n —2 > k n ^i < h. n 



Right-hand side of fl2l)|) 



-7/ 



^n — 2 k n — i 

> fc n _i - 1 



The restrictions for Z„_3 are the same on both sides. The entry l n -2 is involved in a sign-change 
on both sides (note the special case l n -2 — k n —i, where l n -2 is part of a sign-changing pair on the 
left-hand side and a newcomer on the right-hand side). 

It follows that sc(fci, . . . , k n -2, kn-i - l',h, • ■ -, ln-2) = sc(fci, . . . , k n ;h, . . . , l n -l). 
Case 2 (k n _i > k n ) : 

Recursion (|2.5p of the summation operator and the induction hypothesis yield 

(fcl,...,fc n ) 

A(/i,...,i B _i) 

(h,...,i„-i) 

(kx,...,k n -i) k n -x— 1 (fci,...,Je n _a) 
= - ^ ^ A(Zi,...,Z n _i) - ^ A(Zi,...,/ n -3,fcn_i,fcn-i) 

(Jl.-.In-a) in-l=A;„ + l (ll,...,*n-3) 

fc„_l— 1 

E (_ 1 )Bc(fe 1 ,...,fc n _ 1 ;i 1 ,...,i„_ 2 )+l £ . . . , i n -l) 

(ii,...,i n _ 2 )e7'(fci,...,fc n -i) i„_ 1= fc„+i 

+ E (-l) K ( fcl '-"' fc -- ! ' i,l '-"' , --^ +1 A(Zi,...,f n _3,fc n _i,fc rl _i). 

(Ji,...,!„_ 3 )eP(fc I ,...,fc„-a) 

Similarly, let us show that for fc„_i > fc„ 
"P(fci, . . . , k n ) 

= V{ki, fc„_i) x {Z„_! | fc„_! > l n _ x > k n } U . . . , fc„_ 2 ) x {(fc„_i, fc„_i)} (2.7) 

holds. Again it suffices to show that l n —i, l n -2 and l n —a have to satisfy the same restrictions on both 
sides, and that corresponding rows contain the same number of sign-changes. Since k n -i > k n , it 
follows that fc„_i > l n -i > k n on both sides. Let us distinguish between the cases fc n _i > l n -\ > k n 
and l n -i = fc n _i: 

Case 2.1 (k n _i > l n i > k n ) : 
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If fc„_2 > fcn-i, then k n ^2 > ln-2 > fcn-i on both sides: 



Left-hand side of (12~T1) 



I, 



1/ 



-7 



> 



kn — 1 



ln-1 

•A 

> k n 



Right-hand side of $2J\ 



Is 



k n -o 



ln-1 



> 



ln-1 



4 



kn—1 



The restrictions for l n s are the same on both sides. The entries l n -i and l n -i both contribute a 
sign-change. 

If fc n _2 < fcn-i) then fc„_2 < 'n— 2 < k n — i on both sides: 



Left-hand side of d^TTJ) 



1 



^n-2 



< 



k n -i 



ln-1 

-A 

> k Ti 



Right-hand side of (12T7) 



- 



ln-1 



< 



kn — 1 



The restrictions for Z„_3 are the same on both sides. The entry l n -\ contributes a sign-change, and 
the entry l n -i is involved in a sign-change on the left-hand side, if and only if it is on the right-hand 
side. 

It follows that sc(fei, . . . , fc„_i; h, . . . , l n -2) + 1 = sc(fci, . . . , k n ; Zi, . . . , l n -i)- 
Case 2.2 (l„_i = k n _!) : 

Since fc n _i > k n and l n -± = k n —i, we have l n -2 = k n —i on both sides, whereby (7„_2, is a 

sign-changing pair. It remains to be shown that l n -3 has the same restrictions on both sides. 

If fc n _3 < k n -2, then k n s < l n -3 < &n-2 on the right-hand side: 



Left-hand side of 




^n— 3 ^n— 1 


k n —l 






k n -3 < k n -2 


kn—1 - k n 


Right-hand side of 


(12.71) 


ln-3 




h 








k n -3 < k n -2 
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On the left-hand side we also have k n -3 < l n -3 < fcn-2, unless k n ~2 = k n -i- In this case k n ~3 < 
l„-3 < fc n ~2, but for Z„_3 = fc„_2 = fcn-i = In— 2 = ln-i there are three consecutive equal entries 
anyway. The entry l n -3 is involved in a sign-change on the left-hand side, if and only if it is on the 
right-hand side, and (l n -2)ln-i) is a sign-changing pair. 

If fc„_3 > fc n -2 (and n > 4), then fc n _3 > Z„_3 > /c„_2 on the right-hand side: 



Left-hand side of (12.7 



hi-3 



k n -i 



-7/ 



kn~ 



kn—1 



k n —i 



> 



4 



Right-hand side of (l^7|l 



1, 



«n-3 



> 



kn—1 



Again, i n _3 has the same restrictions on the left-hand side, unless fc n _2 = k n -\. In this case 
k n -3 > l n -3 > k„- 2 , whereby Z„_ 3 = fc„_ 2 implies that i„_ 3 = fc„_ 2 = fcn-i = = l n -i- If 
n = 4, then the same holds with the difference that fci > l\ instead of k\ > ii on both sides. The 
entry / n _3 is involved in a sign-change on both sides and (l n —2, ln—i) is a sign-changing pair. 



It follows that sc(fci, . . . , fc„_ 2 ; h, ■ • ■ , in-3) + 1 = sc(fci, . . . , fc n ; . . . , f n -i). 



Proof (Theorem[I\). The result is immediate for n = 1: 

a(l;fei) = l= X! (-!) SC(A) - 
Aeei(fci) 

For n > 2 apply ()2.2|) . Lemma [1] and the induction hypothesis: 



□ 



a(n; fci, . . . , fc„) = E a(n - 1; Zi, . . . , l n -i) 
(h,...,i„-i) 

E (-l) sc(k;1) a(n-l; h,...,l n -i) 

(?l,...,Jn-l)eP(fcl,...,fcn) 

E (- 1 ) sc(k;1) E (-i) sc(A) = E 

(Ji,...,J«-i)eP(ki,...,fcn) Aee„_ 1 (i 1 ,...j„_ 1 ) Aeff«(*i,...,fcn) 



bc(A) 



□ 
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3. Connection with different generalization & Alternative proof 

In [Fisll] four different combinatorial extensions of a(n; kx, . . . , k n ) to all (kx, . . . , k n ) S Z™ are 
described. The idea behind all of them is to write the sum in (|2.1|) in terms of simple summations, 
i.e. summations as defined in (|2.4[) . In the third extension this is based on the inclusion-exclusion 
principle: Let fci < k2 < ■ ■ ■ < k n and 

M := {(h, . . . , l n -x) € Z"- 1 | Vj : kj < l 3 < k j+1 A I, < 

A := {(h, . . . , Z„_ x ) e Z"- 1 | Vj : kj < lj < k j+1 }, 

A, := {(li,...,l n _x) e I/'- 1 | Vj : kj < lj < k j+ i A k-x = h = h}, i = 2,...,n - 1. 

The strict increase implies that Ai D Aj+i = 0, and thus we have for any function f(l) := 
f(h, . . .,Z n -i) that 

£/a) = £/a)- ££/(i)+ £ £ /a) 

1EM leA i=2 leAi 2<ii<i 2 <n-l \£A il nAi 2 

£ £ /(!)•••, (3-1) 

which can be written in terms of simple sums as 

ki k 3 fcij ^ip fe„ 

£M) p £ ££-£ £ - £ £ - £ /0). 

p>0 2<ii<i2<---<i p <n— 1 ii=fci h=k 2 h 1 — l=k il h 1 =k il l ip — l=k ip l ip =k ip i„_ 1 =fc„_ 1 

(3.2) 

Using (|2.4|) . we can interpret (|3.2[) for arbitrary (fci, . . . , fc„) S Z". Let us show that 
a(n;fci,...,Av,)=£(-l)* £ (3.3) 

p>0 2<ii<i 2 <---<i P <n-l 
ij + l#ij+l 

££•••£ £ ••• £ £ ••• £ > o 

ll=kil 2 =k 2 l=fei 1 (i 1 =fci 1 l i p -i=k ip h p =k ip ;„_ 1= fc„_i 

holds for (fci, . . . , fc„) e Z™. The correctness for fci < &2 < • ■ • < is ensured by (12. ip . (13. ip and 
(|3.2j) . To prove it for arbitrary (fci, . . . , fc„) G Z™, it thus suffices to show that (|3.2j) applied to a 
polynomial in Zi, . . . , l n ~\ yields a polynomial in fci, . . . , fc„. But this follows from (|2.4j) in the exact 
same way as in the proof of (|2.2[) . 



As pointed out in [Fisll] . we can give (|3 . 3[) a combinatorial meaning by interpreting a(n; fci, ... , fc„) 
as signed enumeration of the following combinatorial objects: In a triangular array (di,j)x<j<i<n 
of integers, let us call the entries a,i-x,j-i and a,i-x,j the parents of dij. Among the entries 
(tti,j)i<i<i<n> there may be special entries. Special entries in the same row must not be adja- 
cent (choosing these special entries corresponds to fixing the i;'s in (13.310 . The requirements for 
the entries are 

(1) If di t j is special, then a,i^x,j-i — — <H—ij- 
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(2) If aij is not the parent of a special entry and a-i+ij < at+ij+i, then di+ij < a^j < aj + i J+ i. 

(3) If aij is not the parent of a special entry and a,i+i,j > a; + i J+ i, then dj+ij+i > a^j > dj+ij. 
In this case a^.j is called inversion. 

Let us denote by T n (k\, . . . , k n ) the set of these objects with bottom row (a n ,ij ■ • ■ ? a n,n) = (&ii • • • i k n )- 
For A 6 T n {k\ 1 . . . , &„) let s(A) be the total number of special entries and inversions. Using induc- 
tion and (|3.3[) . we thus have 

a(n;k 1 ,...,k n ) = ]T (-l) s(A) . 
We can now eliminate those arrays (cn,j)i<j<i<n violating the condition 

a i,j-l < a i,j < a i,j+l Qi—lJ—l < a i-l,j (3-4) 

by using the following sign-reversing involution: find the minimal index i, and under those the 
minimal index j such that di t j—i < dij < and Oj-i j-i = aj,j = aj-ij. If a^j is special, then 

turn it non-special, and vice-versa. Note that the minimality of i ensures that turning a;j special 
is admissible: Suppose a neighbour of a^j is special, then the row above contains three consecutive 
equal entries and thus an entry violating (13.41) . It follows that 

a(n-k u ...,k n )= J2 (-1) S(A) . 

Note that in this reduced set an entry a^j is special if and only if aj_ij_i = a^j = a,_i.j. Hence, 
the additional information of which entries are special is not required anymore. Since special entries 
now correspond to sign-changing pairs and inversions to newcomers, the only remaining part for 
proving Theorem [T] is to show that 

Gn(kl, . . . , k n ) = {A G Tn(kl,. . . , fc„) I <H,j-l < di,j < Oi,j+l Oj-lj-l < ftj-ij}, 

where an entry a^j is special if and only if aj_ij_i = eijj = a^ij. 

Let ^4 G Q n (ki, . . . , fc„). Then two adjacent special entries in a row would imply three consecutive 
equal entries in a row, in contradiction to Condition @ of GMTs. If aij is special, then a,-_ij_i = 
= a i-ij by definition. If <2j+i,j < &i+i.j+i, then aj+i,j < (Zjj < aj+ij+i by Condition (TTJ of 
GMTs. If dj+ij > a^+ij+i, then aj+ij > a^j > aj+ij+i by Condition |T]) of GMTs, and if <ij+i,j 
and dj+ij+i are neither special, Condition ([3]) of GMTs implies that £ii+i,j > ay > a^+ij+i- We 
thus have A G T n (ki, . . . , and the additional property is exactly Condition @ of GMTs. 

Let A G 7~ n (ki, . . . , k n ) such that <Jjj-i < a»j < ay+i implies aj_ij_i < <ij_ij. Conditions ([I]) 
and ([2|) of GMTs are then trivially satisfied. If ay = Oi+i,j > o-i+i.j+i, then by Condition ([3|) the 
entry ay has to be parent of a special entry, and thus ay — a-i+i.j — ay_i. The second part of 
Condition of GMTs is symmetric, and therefore A G Gn(ki, . . . , k n ). 

This concludes the less straight-forward, yet much shorter proof of Theorem [1] 
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4. Applications & Open Problems 



With this generalization at hand, we can try to give a combinatorial interpretation to identities 
satisfied by a(n; fci, ... , fc„). By way of illustration, take the identity 

a(n; kx,..., h-x, h, ki + 1, k i+2 , . . . , k n ) (4.1) 
= a(n; kx,..., ki-x, ki, h, k i+2 , . . . , k n ) + a(n; kx, ■ ■ ■ , ki-x,h + 1, h + 1, k l+2 , k n ). 

A combinatorial proof of this identity in the case that kx < k 2 < ■ ■ ■ < ki and ki + 1 < ki +2 < ■ ■ ■ < 
k n was given in |Fisll] . Using Theorem [TJ we can now give a combinatorial proof for arbitrary 
(kx, • ■ • , k n ) G Z™ by showing that there exists a sign-preserving bijection 

Qn (kx , ■ • ■ , ki—X, ki, ki + 1, fci+2 , • ■ • , k n ) 

^ Qn (kx , ■ ■ ■ , ki—x, ki, ki, ki+ 2 , . . . , kn) U Q n (kx , ■ ■ ■ , ki— x, ki + 1, ki + 1, ki+ 2 , . . . , &n)- 

If V(kx, ■ ■ ■ ,k n ) denotes the set of penultimate rows of GMTs with bottom row (k\ , . . . , kn) , it 
suffices to show that 

V(kx, h-x,ki, h + 1, k i+2 , ...,k n ) 

= V(ki,. . . , ki-i,ki,ki,k i+2 , k n ) U V(k 1 , fci_i , fcj + 1, h + l,k i+2 , k n ), (4.2) 

where each fixed row has the same total number of sign-changes on both sides. 

Each (lx, . • • , l n -x) £ TQti, ■ ■ ■ , fei-i, ki, ki + 1, ki +2 , . . . , k n ) satisfies k € {ki, ki + 1}. Let us show 
that the set of penultimate rows with Zj = ki is equal to V(kx, ■ ■ ■ , ki—x, ki, ki, ki +2 , . . . , fc„). It is clear 
that li = ki implies that the restrictions for (Zi, . . . , Zj_i) are identical for both V(k\, . . . , ki-x, ki, ki + 
1, ki +2 , . . . , k n ) and V(k\, . . . , ki-x, ki, ki, ki +2 , . . . , k n ). For the restrictions of (Zj+i, h+ 2 ) distinguish 
between ki + 1 < ki +2 , ki = ki +2 and ki > ki +2 : 

If ki + 1 < ki +2 , then fc, + 1 < Zj+i < ki +2 on both sides and the restrictions for U +2 are the same: 



Left-hand side of J42 



< 



1 



Zi+l 



< 



Cj+2 



Right-hand side of (|4~2 



ki 



h+1 



< 



k 



i+2 



If fci = ki +2 , then V(kx, ■ ■ ■ , ki-x, ki,ki, ki +2 , . . . , k n ) is empty, and each element of V(k\, . . . , fc,_i, fci, fci 
1, fci+2, • ■ • , k n ) with Zi = fci would have to satisfy h — li+i = li+ 2 = ki: 
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Left-hand side of 



Y- 



1 



< 



> h 



Right-hand side of (|4~2 



ki 



i 



ki 



But, since a GMT can not contain three consecutive equal entries, there is also no element in 
V{k\, . . . , fej_i, h, ki + 1, ki, . .. ,k n ) with k = k,. 

If ki > ki+2-, then ki > li+\ > on both sides and the restrictions for I1+2 are the same: 



Left-hand side of 


H-2|) 


hi 


£i+l 


// -7 




ki <c ki H~ 1 




Right-hand side of P~2l 




+ 1 






ki — ki > kiJ r 2 



The entry is involved in a sign-change on both sides (note the special case U+\ = ki, where 
U+i is a newcomer on the left-hand side and in a sign-changing pair on the right-hand side). 

The restrictions for (h+3, ■ ■ . , ln-i) are clearly the same for both sides. Symmetrically, one can 
also see that the set V(k\, . . . , fej-i, ki, ki + 1, ki+2, ■ ■ ■ , k n ) restricted to £, = ki+\ is the same as 
V(k\, . . . , fei—i, ki + 1, ki + 1, ki+2, ■ ■ ■ , k n ), concluding the combinatorial proof of (|4.ip for arbitrary 
(*!,..., fen) £ Z". 

A natural question could now be, whether similar identities hold if the difference between ki+i and 
h is larger. For fixed integers ki, . . ■ , fci+2, • • • , k n , let 

(ki , ki+l) '.= CH[Tl; fel , . . . , ki— 1 , fej, , fcj_|_2 , • ■ • , feji)- 

Similarly - with a bit more patience - one can also show the identity 

tn{ki, ki + 2) 

= t n (ki, h) + t n (ki + 1, ki + 1) + i„(fc l + 2,h + 2) + t n (ki + 2, A* + 1) + t n {h + 1, h) (4.3) 



combinatorially. Both (|4. 1[) and (J4T3J) are special cases of the following identity: Let V x ,y be the 
operator defined as 

V x , y f(x, y) := f(x-l,y) + f(x, y + 1) - f(x - 1, y + 1). 
The function fi(ki, . . . , fc„) := V^^^a^n; k\, . . . , k n ) then satisfies 

/i(fei, ■ ■ • ,k n ) = -fi(h, . . .,ki-i,k i+ i + 1, fc l+2 , ■ ■ ■ ,k n ). (4.4) 
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Setting ki+i = ki — 1 in (|4.4[) immediately implies (|4.ip . Equation (|4.3I) is then the special case 
fcj+i = fcj — 2 in (14. 4p . A similar shift- antisymmetry property for Gelfand-Tsetlin Patterns (Mono- 
tone Triangles without the condition of strict increase along rows) was shown bijectively in a recent 
work ( jFisllj ). It would be interesting to give a bijective proof of (|4.4|) in the general case (an 
algebraic proof was given in [Fis06] ) . 

In FR11 we showed the surprising identity 

An := a(n; 1,2, ... ,n) = a(2n; n, n, n — 1, n — 1, . . . , 1, 1) (4-5) 

algebraically and gave initial thoughts on how a bijective proof could succeed. Let us conclude 
with a list of related identities - all of them are up to this point conjectured using mathematical 
computing software. As Theorem [1] provides a combinatorial interpretation of these identities, 
bijective proofs are of high interest. 

Conjecture 1 ( [FRllj ). Let n > 1. Then 

a(2n + 1; 2n + 1, 2n, . . . , 1) = {-l) n a{n; 2,4,..., 2n) (4.6) 

seems to hold, whereby a(n; 2,4,..., 2n) is known to be the number of Vertically Symmetric ASMs 
of size 2n + 1. 

Conjecture 2. Let n > 1. Then 

a(n; 2, 4, . . . , 2n) = a(2n; 2n, 2n, 2n - 2, 2n - 2, . . . , 2, 2) (4.7) 

seems to hold. 

Conjecture 3. Let n > 1. Then 

A n = a(n + i; 1,2, . . . ,%, 1, 2, . . . , n), i = 0, ...,n, (4.8) 
An = (-l)"a(2n + 1; 1, 2, . . . , n + 1, 1, 2, . . . , n) (4.9) 
seems to hold. Furthermore, the numbers 

W n ,i = a(2n + 1; i, 2, . . . , n + 1, 1, 2, . . . , n), i = l,...,3n + 2 
seem to satisfy the symmetry W n .i — W n ,Sn+3-i- 
Conjecture 4. Let n > 2. Then 

An=a(n + 2;l,2,...,i + l,i,i + l,...,n), i = l,...,n-l (4-10) 

seems to hold. 

Further computational experiments led to the conjecture that (|4.5[) and (|4.10p have the following 
joint generalization: 

Conjecture 5. Let n > 1. Then 

A n = a(n + k; 1, . . . , i — 1, i + k — 1, i + k — 1, i + k — 2, i + k — 2, . . . , i, i, i + k, i + k + 1, . . . , n) (4.11) 

seems to hold for i = 1, . . . , n — k + 1, k — 1, . . . , n. 

In words, the last identity takes a subsequence {i, i + 1, . . . , i + k — 1) of length k of (1,2, ... ,n), 
reverses the order, duplicates each entry and puts the subsequence back. Identity (|4.5[) is thus the 



GENERALIZED MONOTONE TRIANGLES 



17 



special case of (j4.11j) where k = n. Applying (14+j) and the fact that a GMT can not contain three 
consecutive equal entries, shows that (|4. 10[) is the special case of (|4.11| with k = 2: 

a(n + 2; 1, 2, . . . , i — 1, i, i + 1, i, i + 1, i + 2, . . . , n) 
= a(n+2; 1, 2, . . . , i — 1, i, i+1, i+2, . . . , n)+a(n+2; 1, 2, . . . , i — 1, i+1, i+1, i, i+1, i+2, . . ., n) 
= a(n+2; 1, 2, . . . , i— 1, i+1, i, i, i+2, . . . , n)+a(n+2; 1,2, . . . , i— 1, i+1, i+1, i+1, i+1, i+2, . . . , n) 

= a(n +2; 1,2, ...,i- l,i + l,i+ l,i,i,i + 2, ...,n). 

From the correspondence between ASMs of size n and Monotone Triangles with bottom row 
(1,2, ... ,n), it follows that a(n — 1; 1, 2, . . . , i — 1, i + 1, . . . , n) is equal to the number of ASMs 
of size n with the first row's unique 1 in column i - denoted A n i . In the following conjecture we 
analogously remove the i-th argument of the right-hand side in (I4.10[) : 

Conjecture 6. Let n> 1. Then 

n 

a(n+ 1; 1,2, ...,i- l,i + l,i,i+ l,...,n) = -^(j - i)A n . } , i = l,...,n- 1 (4+2) 

3=1 

seems to hold. 

As a note on how we found (|4.12[) , let us prove the case i = 1: Each penultimate row (h, . . . , l n ) 
of a GMT with bottom row (2, 1, 2, . . . , n) satisfies h = l 2 = 1 by Condition © of GMTs. Taking 
Conditions ([T]) and ([2j into account, Lemma [1] implies that 

n 

a(n + l;2,l,2,...,n) = - a(n; 1, 1, 2, . . . ,p - l,p + 1, . . . ,n). 

p=2 

Each penultimate row (mi, . . . , m„_i) of a GMT with bottom row (1, 1, 2, . . . ,p — l,p + 1, . . . , n) 
satisfies toi = 1, TO2 = 2, . . . , rn p -i = p — 1. Applying Lemma [T] again yields the claimed equation: 

n n n 

a(n + 1; 2, 1, 2, . . . , n) = - ]T £ + nj = £(i ~ i)^- 

For general i, the set of GMTs with bottom row (1, 2, . . . , i — 1, i + 1, i, i+ 1, . . . , n) can be written 
as disjoint union of those with structure 

51 : h ■ ■ ■ U-2 i + 1 i + 1 « h+2 ■■■ l n 

1 • • • i-2 i-1 i + 1 i i + 1 i + 2 ••• n, 

5 2 : li • • ■ <i-3 i + 1 i i ii + 2 ■ ■ • i n 

1 ■•• i-2 i-1 i + 1 i i + 1 i + 2 ••• n, 

53 : h ■ ■ ■ h-2 i — 1 i i ii+2 ■ ■ • in 

1 •■■ i-2 i-1 i + 1 i i + 1 i + 2 ■■■ n. 

Similar to the case i = 1, one can see that the signed enumeration of GMTs with structure S3 is 
equal to 

n 

j=i+l 
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Proving that the signed enumeration of GMTs with structure Si and S2 yields 

i-l 

i=i 

remains an open problem. The following conjectures are also related to (|4. 10[) by removing the 
(i — l)-st argument of the right-hand side. 

Conjecture 7. Let n > 4. Then 

n + 4 

a(n + 1; 1, 3, 4, 3, 4, 5, . . . , n) = -g-A-i (4.13) 

seems to hold. 



As an immediate consequence of Theorem [T] we obtain (the known fact) that the evaluation of 
a(n; k x , . . . , k n ) at integral values is integral. From the definition of ASMs it follows that Vertically 
Symmetric ASMs only exist for odd size. Therefore, reflection along the vertical symmetry axis is 
a fixed-point-free involution on the set of even-sized ASMs. So, the number of even-sized ASMs is 
even and the the right-hand side of (I4.13|) is an integer too. 

Using C. Krattenthaler's Mathematica package RATE, we were able to find more conjectured 
formulas similar to (14. 13)) : 

Conjecture 8. 

n 1 n t r a r \ n 3 + 7n 2 + l0n-36 A 
a(n + 1; 1,2,4,5,4,5, ...,n) = g ^ _ u A n - X , n > 5, 

r^ 119 ,c K r fi \ n 4 + 12n 3 + +53n 2 + 54n - 288 
a(n+ 1; 1,2, 3, 5, 6, 5, 6,..., n) = ^ - ^ A n - X , n > 6. 



In general, this leads to the following conjecture: 

Conjecture 9. Letn > k > 4. Then there exist polynomials pk{n) andqk(n) with deg pk — deg qk = 
k — 3 such that 

a(n + 1; 1, 2, . . . , k — 3, k — 1, k, k — 1, fc, . . . , n) = ^ k . A n _i- 

qk(n) 
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